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It is generally understood that the mission of the CIA is to provide intelli- 
gence to the President, his administration and the Congress—data collec- 
tion, analysis, forecasting and interpretive reporting.. The product desired 
by the administration is, presumably, either the best available estimate (as 
opposed, for example, to a “‘worst case’’ scenario) or a range of outcomes 
to which probabilities are attached. 

The committee was charged with an evaluation of the CIA’s published 
work from approximately 1970 through 1990 with particular regard to the 
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quality of estimates of Soviet GNP and rate of growth, the size and burden 
of Soviet defense expenditures, and U.S.-Soviet comparisons. The Review 
Committee was also charged with evaluating the CIA’s appraisals of the 
prospects for change in the Soviet economy, especially their success in antici- 
pating change following Brezhnev’s death, and their accuracy in interpret- 
ing reform and its prospects. Finally, the Committee was asked to provide 
evaluation of CIA’s interpretive reporting as it pertained to the Soviet Union. 


Performance Evaluation 


To evaluate the CIA’s performance against a reasonable and objective 
standard, we selected the performance of academic Soviet specialists or, 
more generally, ‘‘best practice’’ methodology. Clients and the public may 
have unrealistically high expectations about just how good the CIA’s reports 
and forecasts ought to be. Neither the CIA nor academic specialists were 
particularly percipient, for example, in anticipating the outcome of 
Gorbachev’s policies of perestroika and glasnost. The research community 
as a whole failed to consider the collapse of communist as a possible 
scenario. If the CIA was expected to do a better job, then on this criterion 
the Office of Soviet Analysis (SOVA) failed, but we believe this is too strict 
a criterion. In fact, we believe that much of the criticism is based on twenty- 
twenty highsight. In our view, the best that can be expected is for policy 
makers to be given a range of outcomes or a set of scenarios to which broad 
probabilities are attached. This allows policy makers to become aware of 
the various options and to prepare contingency plans accordingly. Such an 
approach is also more forthright about the difficulties of forecasting eco- 
nomic outcomes generally. 

We conclude in general that the work of the Office of Soviet Analysis 
and its predecessors in the CIA has been professional and appropriately 
reasonable and cautious. While we find no indication of systematic mis- 
representation, incompetence or carelessness, we have serious reservations 
about certain methodologies and reporting styles of the defense group. 

Although the CIA did fail to predict the catastrophic developments in the 
Soviet economy at the end of the 1980s, it did begin to alert its clients to 
a serious and continuing slowdown in the Soviet economy and an increas- 
ing competition for resources much earlier. A 1977 report (ER 77-10436U), 
for example, predicted accurately that a reduced rate of economic growth 
was ‘‘inevitable’’ for the 1980s and would make it very difficult and pos- 
sibly impossible for Moscow to pursue three traditional objectives of the 
leadership: (a) catch up with the U.S. militarily, (b) expand the industrial 
base steadily and (c) meet at least minimal consumer expectations for 
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improved living standards and welfare. Subsequent reports amplified this 
continuing problem. A 1979 report (ER 79-10131) forecast with impres- 
sive accuracy that the Soviet economic bureaucracy had ‘‘surprisingly little 
maneuverability” and would ‘‘be hard pressed to maintain growth at an 
average of 3 percent a year [for 1980-85].’’ Reporting again in 1981 (ER 
81-10009), the CIA stated that the resource crunch was becoming so great 
that ‘‘something has to give.’’ Most reports through 1988 on the course 
of Soviet GNP and on general economic developments were equally satis- 
factory: accurate, illuminating and timely. 

In fact, we find it difficult to believe that anyone who has read the CIA’s 
annual public reports on the state of the Soviet economy since 1975 could 
possibly interpret them as stating that the Soviet economy was ‘‘booming.”’ 
On the contrary, these reports regularly reported the steady decline in the 
Soviet growth rate and called attention to the deep structural problems that 
pointed to continued decline and possibly to stagnation. As early as 1982, 
SOVA analysts were evaluating the potential contribution of the conver- 
sion of defense plants to civilian production as a means to offset economic 
decline (Sov 82-10101). SOVA also monitored closely the internal Soviet 
debates on economic reform and correctly predicted that the Brezhnev 
government would not undertake the kind of radical economic transforma- 
tion required to restore vigor to the economy (Sov 82-10068). Andropov 
was described as committed to ‘‘continuity and moderation’? (Sov 
83-10115), and Gorbachev was correctly identified early on as a determined 
reformer (Sov-85-10165). What remained unknown and perhaps unknow- 
able, was whether the Soviet economy was reformable. 

The review committee does believe, however, that the performance of the 
CIA can be improved and, just as importantly, that recent changes in the 
Soviet Union and Eastern Europe require a respecification of the tasks SOVA 
should be asked to carry out. We also believe that steps need to be taken 
to integrate SOVA’s work with that of other national and international agen- 
cies which now have responsibility for following Soviet developments. 

Our most significant critical conclusion is that certain CIA products dur- 
ing the 1970s and 80s may indeed have tended to overstate the size of the 
Soviet economy relative to the U.S. and, particularly, its military threat. 
The CIA’s main quantitative research reports during these years may be 
divided into two broad categories: (1) direct measurement and evaluation 
of Soviet economic capacity and performance and (2) comparisons of Soviet 
and U.S. production capacity and economic performance. In our view, 
most of what is legitimate in recent criticism of the CIA springs from the 
comparative work. That is, if the CIA had confined itself to the analysis 
of Soviet economic and military performance over time and refrained from 
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comparative assessments, the CIA’s reports might very well not have gener- 
ated the current controversy. It is the review committee’s conviction that 
the U.S.-Soviet comparisons, say of size of GNP and dollar-volume mili- 
tary expenditures, rest on questionable methodological foundations, and yet 
these measures apparently have had great intuitive meaning and thus in- 
fluence with clients and the public alike. Whether or not this contributed 
to an excessive and unnecessary U.S. military buildup is not a matter the 
review committee is qualified or has the resources to address. 

Finally, we believe that an opportunity exists today, thanks to the dramatic 
developments in the Soviet Union over the last several years, for the CIA 
to reorganize SOVA. We recommend a reconsideration of SOVA’s func- 
tions in the light of the activities of the World Bank, the IMF, the UN, 
the OECD and the EBRD which are now involved in measuring and ap- 
praising Soviet economic performance. There may even be opportunities 
for SOVA analysts to work directly with Soviet Union and republican agen- 
cies. Moreover, there is a clear need to draw analysts into SOVA who are 
sophisticated in economic modelling and in a wide variety of institutional 
and other qualitative forms of analysis. 


Measures of Soviet GNP and Consumption in Rubles 


The CIA’s estimates of Soviet GNP in rubles are based upon an adjusted 
factor cost (AFC) standard pioneered by Abram Bergson and his students 
in the 1950s. With the exception of the defense expenditure component, 
these estimates are based predominantly upon open sources. The method- 
ology underlying the estimates has been presented periodically to the schol- 
arly community for review, criticism and recommendations. There has been 
very little criticism of this work, and CIA estimates have been accepted 
as authoritative throughout the world, including the Soviet Union. Recog- 
nizing that AFC estimates do not take into account variation in hidden 
inflation rates across sectors and do not reflect differences in capital produc- 
tivity, the end-use and sector-of-origin breakdowns have proven invaluable 
for analysis. 

Consumption in rubles is derived from the GNP estimates. There has 
been little criticism of these numbers. Their main value is in providing a 
sense of consumption’s share in GNP as a basis for speculation about wel- 
fare, morale, incentives and so forth. Neither the GNP nor the consump- 
tion estimates, however, include ‘‘second economy” activities. While the 
committee realizes that routinely providing estimates of the size and scope 
of second economy activities would have been impossible, some assess- 
ment of its impact, even if speculative, was warranted. Such an evaluation 
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would also have served to lay the foundation for analysis of the impact of 
the growing cooperative and commercial sector activities in the late 1980s. 

SOVA should emphasize in its future publications that its Soviet GNP 
figures measure, at best, production potential. Unlike GNP measures in 
market economies where economic activity is guided by decisions by firms 
and households that enhance their material well-being, Soviet GNP meas- 
ures summarize aggregate production activities regardless of whether the 
goods produced satisfy demand. The literature abounds with descriptions 
not only of excess inventories of producer and consumer goods, but also 
of a growing level of unfinished construction. To date there has been no 
way to calculate a measure of Soviet GNP that would be a more satisfac- 
tory measure of social welfare. A welfare-based measure of GNP and its 
components would provide a much more satisfactory basis for the analysis 
of welfare implications and as a predictor of political developments. If the 
Soviet economy is producing heavy machinery that no one needs or wants, 
the capacity to do so is recorded in the factor-cost adjusted evaluation of 
production potential, but this clearly overstates the welfare content of such 
products. Recent developments in the Soviet Union may make it possible 
to use market prices in GNP calculations. Current data collection and meas- 
urement activities by international (and Soviet) agencies may indeed make 
the CIA’s effort to measure GNP and its components redundant. 


GNP Growth Rates 


SOVA calculates GNP growth rates using a time series of samples of 
physical production. The data are weighted by share in the base year’s GNP. 
Data limitations and method have been explained fully in open publications. 

The main findings have stood the test of time quite well. The series shows 
relatively rapid growth, but at a declining rate between 1965 and 1975. 
The rate of growth continues to decline from 1976 through 1985, failing 
to approximately 2% per year in the later years. The picture presented by 
SOVA figures is of a once vigorous economy becoming increasingly slug- 
gish for two main reasons: (a) continued reliance on extensive growth be- 
cause of systemic barriers to technological progress and (b) poor investment 
decisions. Thus, the leadership of the 1980s was faced with difficult choices 
posed by the rising demand for both guns and butter in a period of declin- 
ing economic growth. 

SOVA’s growth estimates have been challenged from two directions. 
Michael Boretsky argues that the use of physical time series understates 
quality improvements in Soviet products and therefore understates the rate 
of growth. The implication is that the Soviet economy was, say in 1980, 
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larger and stronger than the CIA reported. The committee agrees with the 
CIA’s response to Boretsky, namely that SOVA did take quality changes 
into account where possible. While there may have been some quality change 
for which adjustments were not made, the understatement is unlikely to 
have been substantial. Any attempt to adjust further for quality change would 
require subjective and nonreproducible evaluations. 

The other challenge is from recent estimates by Soviet economists, notably 
Gregory Khanin. His work was given great publicity at an American 
Heritage conference which, in turn, served as one stimulus to public criti- 
cism of the CIA’s products. A subsequent academic conference convened 
by the CIA examined the issue. We concur with the conclusions of that 
conference: Methodologically, Khanin’s approach is naive, and it has not 
bee possible for others to reproduce his results. Until more detail is avail- 
able, Khanin’s results cannot be regarded as serious competition to CIA 
GNP estimates. 


SOVSIM Modeling 


SOVSIM represents one of the first attempts to set up an econometric 
model of the Soviet economy.” SOVSIM portrays a ‘‘supply-oriented”’ 
centrally planned economy. Thus, binding resource constraints such as 
energy and labor are critical. There is no accounting for technical progress 
based on endogenous factors such as learning by doing, consumer satisfac- 
tion, investment in human capital, or improvements in the capital vintage 
profile. The model also does not provide for optimizing behavior by plan- 
ners, and the structural parameters are exogenously determined. Finally, 
no provision is made for structural change over time. 

For these reasons, the SOVSIM model is not useful for analyzing or 
projecting current developments in the Soviet Union, and we recommend 
termination of SOVSIM modelling. The committee believes, however, that 
the model was useful during the Brezhnev years (the ‘‘period of stagna- 
tion’’) precisely because structural change was so slight. The model predicted 
with impressive accuracy that there would be serious problems by 1985.° 
In particular, given falling oil production, it was predicted that there would 
be a slowdown in growth and that the Soviet Union would be obliged to 
reduce the export of petroleum to Eastern Europe at subsidized prices. The 
model predicted also that, without improvements in oil production tech- 
niques or substitution by natural gas, the Soviet Union would be obliged 
to import oil from the West, with a resulting hard currency crunch. These 
projections were not precisely correct but, in hindsight, seem to the com- 
mittee to have been useful predictions*. Certainly, they do not suggest a 
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healthy outlook for the 1980s. Moreover, had SOVA used a range of 
scenarios, the prediction would have bracketed the actual outcome. 

In a period of rapid structural change it would be good to see more in- 
stitutional and theoretic analyses of the engines of growth. In particular, 
since so much of the high-level physical and human capital is concentrated 
in the Machine Building Metal Working Sector, could this sector provide 
spillovers to other sectors if the wall of secrecy comes down? To what ex- 
tent have defense expenditures driven Soviet economic growth? Will the 
private sector provide an engine of growth, or will its growth along with 
the consequent decline of the state sector retard growth prospects? These 
are the kinds of questions which need to be addressed in a period of rapid 
structural change, and the modern theory of endogenous economic growth 
would seem to be highly relevant too. 

SOVA has had an inclination to present numbers unaccompanied by in- 
depth interpretive analysis. It is a lot easier to crunch numbers than to ana- 
lyze qualitative, structural or organizational change. Generally lacking in 
SOVA reports is the use of microeconomic analysis: principal/agent models, 
transactions costs analysis, or a property rights framework, for example, 
as a means for evaluating the information collected. There is the distinct 
need in SOVA for more individuals with professional experience and depth. 
To attract such scholars, SOVA would need to create conditions (less 
secrecy, more publication opportunities, more open interaction with other 
government departments and agencies, peer review, and so forth) that would 
attract more highly qualified analysts. 


SOVIET-U.S. Comparisons 


Quantitative comparison of economies is an inexact science, with results 
that are highly sensitive to the method used. SOVA’s Soviet-U.S. GNP com- 
parisons are based upon the calculations of price ratios that reflect ‘‘pur- 
chasing power parity’? (PPP) exchange rates, which is best practice 
methodology. The result is a comparison of the relative size of the two 
economies in either rubles or dollars for any given year. SOVA pub- 
lications do point out the limitations and qualifications of the method- 
ology: the results tend to overstate the dollar-value of Soviet GNP and 
to understate the ruble value of U.S. GNP because (1) the higher qual- 
ity, (2) the greater durability and (3) the greater degree to which U.S. 
machinery products are customized (that is, produced in smaller runs) are 
not reflected in the PPPs. No adjustment for these biases are apparently 
deemed feasible, which cannot but severely limit the usefulness of the 
comparison. 
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Two other interpretive problems that SOVA has not stressed adequately, 
in our opinion, have contributed to public criticism of SOVA’s conclusions. 
First, SOVA’s measure of Soviet GNP is a measure of production poten- 
tial, not of social welfare derived from production. Moreover, U.S. prices 
reflect not only domestic well being, but world-market competitive condi- 
tions as well. The value of Soviet output in U.S. prices, therefore, over- 
states the (world) market value of Soviet GNP. 

The index number problem poses a second source of possible miscon- 
struction of SOVA’s GNP comparisons. As SOVA clearly states, for ex- 
ample, estimates of PPPs for investment expenditures in U.S. weights yields 
a higher ruble-dollar ratio (0.63) than Soviet weights (0.40). A similar rela- 
tionship obtains for GNP comparisons. This is a well-known problem of 
which SOVA is well aware. The review committee, however, is critical 
of the fact that SOVA elected to present its GNP comparisons as a single- 
valued geometric average. We believe that it should have resisted any pres- 
sure to simplify its results in this way and should instead have presented 
at all times both the ruble and the dollar comparisons. To report that the 
average ratio of Soviet to U.S. GNP was 60.4% in 1976 implies a degree 
of precision much greater than a report stating that the ratio is 49.5% when 
measured in rubles or 73.8% when measured in dollars. In any event, all 
members of the committee agree that no one of these numbers by itself con- 
veys the information that is supplied by all three. In this respect, it should 
be emphasized that the relative size of the Soviet and U.S. economies is 
not a single ‘‘knowable’’ number. Inherently, there is more than one answer 
to the question of size because one cannot exclude either set of weights 
on a pfiori grounds. 

Despite the limitations noted, SOVA’s comparison of Soviet and U.S. 
GNP for 1960-83 looks quite good in retrospect if viewed as a measure 
of trends. We would argue in hindsight that as a measure of social well 
being the numbers are too high. However, the trend is correct and a valu- 
able finding. 

Comparisons of Soviet-U.S. total and per capita consumption raise simi- 
lar methodological issues. The 1981 comparison of Soviet and U.S. con- 
sumption by Gertrude Schroeder and Imogene Edwards (**‘Consumption in 
the U.S.S.R.: An International Comparison’’) merits special mention in 
the opinion of the review committee. The study was modelled upon the In- 
ternational Comparison Project (IPC) that was developed by Irving Kravis 
et al. under sponsorship of the World Bank and the UN. Schroeder and 
Edwards compare Soviet and U.S. consumption levels in 1976 by using 
a sample of 334 goods and services. The study matches U.S. and Soviet 
products and develops a ‘‘binary comparison of per capita consumption of 
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goods and services of a quantitative nature, resulting from valuation of both 
countries’ consumption in 1976 dollars and rubles and taking the geometric 
mean of the comparison in dollars and the comparison in rubles. ”’ 

Methodologically, there are some unavoidable problems with this ap- 
proach. It is difficult to do a binary comparison when there is much variety 
in the U.S. and so little in the U.S.S.R. Items that are identified as 
‘‘matches”’ are likely to be more representative of the Soviet than the U.S. 
product mix. As the study noted, the fact that high quality and thus high- 
priced U.S. products are thus likely to be excluded, for example, tends to 
bias the results in favor of the U.S.S.R. Further, the Soviet Union is a sellers’ 
market, which means that rubles do not have the claim on goods that dol- 
lars have in the United States. Finally, it is not clear why a geometric mean 
is used. Indeed, it would be valuable in our opinion to explore the causes 
of the large difference that obtains between the ruble and dollar measures. 

Despite these problems, we believe that the study is very valuable. The 
authors are careful to point out the limitations of their methodology. An 
important conclusion of the study is that as of 1976, real per capita con- 
sumption in the U.S.S.R. was 34.4% of that in the U.S., and the lag in 
consumer and household services was massive. We conclude that a careful 
reading would tell a policy maker that the Soviet economy was in trouble 
already in 1976. 


Measures and Comparisons of Soviet Defense Expenditures 


In the estimates of GNP, the bulk of the data are taken from published 
Soviet sources, and the agency presents its raw data and its methodology 
in great detail. Its findings are therefore subject to evaluation by independent 
analysts in a manner fully consistent with the highest scholarly standards. 
Defense estimates, however, are derived from highly classified sources. 
The CIA publishes the findings, but presents only a general account of the 
kind of data and the methods used to obtains its results. Except by invita- 
tion, outside analysts cannot scrutinize the basic data or assess the estimat- 
ing procedures in detail. Thus, the findings can be assessed by the profession 
at large only by an evaluation of the reported methods of estimation and 
by their broad consistency with better understood economic data. The results 
of reviews that have been commissioned by the CIA have not been widely 
disseminated and are generally not part of the public scholarly domain. The 
defense estimates must, therefore, be regarded as much more speculative 
than the GNP estimates and, therefore, more subject to controversy. 

Our assessment focuses on four aspects of the defense expenditure esti- 
mates: method of estimation, coverage of defense-related expenditure, 
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calculation procedures, and presentation of results. We deal primarily with 
defense procurement estimates; the estimation of current operations and 
RDT&E is not discussed here. 


Method 


Since Soviet national income reporting could not be trusted to provide 
a full measure of total defense expenditures, some indirect method had to 
be developed. Two approaches predominate. One is to rely on published 
monetary aggregates (on value of industrial production, for example) and 
to try to cull out of those figures the unknown proportion that consists of 
defense items (for example, the value of weapon systems in the total of 
industrial production). The other is to collect data on the production of 
weapons and then to evaluate the physical production data at the prices that 
the military purchaser pays for its procurement. The CIA uses the latter 
method, usually called the ‘‘building block’’ approach. 

Each method has its merits and demerits, which have been explored and 
debated at great length for decades and have long been quite fully under- 
stood. In our judgement the balance is in favor of the building block method. 
Flawed as it is, there has never been a strong case for the CIA to abandon 
it because no clearly superior alternative has been discovered. 

One disadvantage is the uncertainty associated with the building block 
method for certain products. It is difficult, for example, to estimate the 
production rate of some weapons from such evidence as satellite photos 
of freight car loadings or manufacturing floor space. In these cases there 
is undoubtedly a large element of subjective judgement required. The proce- 
dure does, however, work well with large items such as ships. It also offers 
a source independent of Soviet official data. On balance, we guess that the 
method is more likely to undercount than to overcount, implying that the 
CIA defense estimates may be too low. We can make no judgement on the 
order of magnitude, however, nor on the importance of this matter relative 
to the many other sources of bias. 

Current changes in the U.S.S.R. may make it possible to obtain more 
direct and reliable information in the future. We recommend that the CIA 
explore with the Soviets ways of exchanging defense information that might 
render the building block method obsolete. It may also be possible to work 
more closely with Soviet scholars (Sergei Rogov and Andrei Kortunov, for 
example) who are calculating Soviet defense expenditures in both defense 
rubles ($1 to 1 defense ruble) and civilian rubles ($0.03 to 1 civilian ruble). 
For the next several years, however, it would be desirable to continue 
the annual estimation procedure in order to be able to detect changes in 
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military spending on the basis of this long time series produced with a 
standardized methodology. 


Coverage 


The agency distinguishes three concepts of defense expenditures. One 
is the U.S. concept, consisting of those items that are classified as defense 
under U.S. budgetary procedures. The second is a ‘‘Soviet’’ concept, com- 
prising a somewhat larger set of activities in the U.S.S.R. It includes such 
items as logging and roadbuilding by troops, internal security forces, and 
civilian space activities. It has been the practice to present two estimates, 
one for each of these two concepts. The difference between the two esti- 
mates has ordinarily been about 1% of GNP. 

Critics have poined out that even the broader concept does not fully cap- 
ture the extent of militarization of Soviet society; for example, civil defense, 
strategic reserves of grain and petroleum, and so forth. Some of the charges 
that the CIA understates Soviet defense expenditures derive from consider- 
ation of this third concept. 

There is some justice, therefore, in the claim that the CIA’s usual cover- 
age of the first two concepts misses some defense-associated expenditures 
and is, therefore, too low for some analytic purposes. Considering this third 
concept, the full defense burden may, therefore, be greater than reported. 
Because of data limitations, however, and in the interest of minimizing sub- 
jectivity in the calculations, we think the agency is correct in sticking to 
its usual practice with respect to coverage. 


Calculations 


While we endorse the CIA’s choice of method and of concept of expen- 
diture, we have found a great deal to criticize with respect to the calcula- 
tions. Given the time constraints under which the review committee worked, 
we found it impossible to obtain an unambiguous understanding of the 
methodology and procedures followed in the calculation of defense expendi- 
tures. It was not clear from the available readings, for example, that the 
base year figures have been revised annually so that in fact there are no 
true base year prices and quantities in the defense expenditure calculations. 
While this procedure was clarified for us by SOVA analysts responsible 
for preparing the defense estimates, the implications of this estimating proce- 
dure on estimates of the defense burden and threat remain elusive. In short, 
we found a number of anomalies, which we failed to clarify from either 
the written documents or in our discussions with SOVA defense specialists. 
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Our doubts came to the fore in our effort to assess the various major revi- 
sions that the CIA has made in its estimates over time. In a number of im- 
portant cases, revision led to an increase in estimates of the defense burden, 
when one would have expected, on theoretical and other grounds, that the 
estimated burden should have decreased. 

The estimating procedure is exceedingly intricate, and in the time avail- 
able to us we were unable to assess the theoretical underpinnings of the 
procedure, or to assess the sources of error that might be expected at each 
of the several steps in the process. In the end we came away impressed 
with the ingenuity of the effort to find a systematic way of estimating a 
mass of unknown information from a small body of known information. 
Because of the many assumptions and estimates that have to be made along 
the way to get the method to work, however, there are many sources of 
possible error in the final estimate of military expenditures. We are con- 
cerned that, in the absence of a reliable independent check, the estimate 
could be wildly wrong without our having any way of knowing so.’ More 
specifically, consider the following three revisions: 

The 1981 Revision. In the 1980 Hearings before the Joint Economic 
Committee, the CIA reported the 1979 defense burden as ‘‘12 to 14%” 
of GNP. As in all preceding years, the burden was presented as a two-percent 
range. There was a clear explanation that the ‘‘lower band’’ represented 
the U.S. definition of defense and the ‘‘upper band’’ represented the some- 
what broader Soviet definition. 

In the 1981 Hearings, however, it was reported that the defense burden 
had risen from its earlier level of ‘12% to 13%”’ to a higher (1980) level 
of “13 to 14%.” No indication was given that the agency had shifted from 
a two-percent range to a one-percent range in reporting the burden, nor 
was any explanation given for the change. 

The review committee has since learned that the CIA had decided in 1981 
to stop reporting the (lower) U.S. definition of defense and to report only 
the Soviet (higher) definition of defense, as they have been doing ever since. 
They point out, correctly, that the Soviet definition is more relevant to the 
assessment of the cost of defense to the Soviet economy. But since the po- 
litical debate inevitably leads to a comparison of how much the Soviets are 
spending compared to how much we are spending on defense, dropping 
the U.S. definition results in relative exaggeration of the Soviet burden. 
Moreover, the introduction of the change in definition without informing 
Congress that there had been such a change, and what its significance was, 
strikes us as a very bad reporting technique. We believe that this episode 
contributed to the public impression that the CIA was presenting its figures 
in such a manner as to give an exaggerated impression of how Soviet defense 
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expenditures were changing. We recommend that both concepts be presented 
separately, in the form of moving averages. Reflecting best practice metho- 
dology, reporting moving coverages would improve the clarity of the report 
and offer a less arbitrary method of smoothing year-to-year variations. 

The 1983 Revision. During the years 1976-81 the CIA reported defense 
spending to have grown at ‘‘about 4-5 percent per year’’ (1981 Hearings). 
In 1982 it reported that defense spending had been rising at an annual rate 
of 4% ‘‘that had prevailed since the mid 1960s” (1982 Hearings). Then, 
in the 1983 Hearings, the CIA announced that, on reviewing the accumu- 
lated evidence, the 1976-81 estimates had been too high. The growth of 
defense expenditures had in fact slowed down substantially after 1976. In- 
stead of the 4-5% per year they had been reporting since 1976, defense 
spending had grown at only 2% per year. However, because GNP growth 
had slowed down along with defense growth, the defense burden ‘‘remained 
roughly constant at 13-14% through the 1970s.” 

Many questions have been raised by this major revision. We consider 
only two. First, the defense burden for 1965-75 had for many years earlier 
been reported as 11-13%. The 1983 Report states, however, that the defense 
share ‘‘throughout the 1970s’’ remained constant at 13-14%. It was not 
evident why the revision of the 1976-81 defense spending estimates should 
have caused the burden estimates for 1970-76 to have been revised upward 
from 11-13% to 13-14%. We were informed that the change resulted from 
revisions in both base year prices and quantities that were not adequately 
explained by SOVA in response to outside challenges. The defense share 
estimates have been confusing because it was not clear to most outside ob- 
servers that the entire time series is recalculated each year. We are sym- 
pathetic to this unusual procedure, given the objective, but we believe that 
it could have been explained much more simply than has thus far been the 
case. 

Second, it would seem that if the growth rate of defense spending had 
been overstated during the years 1976-80, the burden must also have been 
overstated. That is, if a numerator (defense spending) is reduced but a 
denominator (GNP) is unchanged, the ratio (defense burden) must be 
smaller. In the 1983 Hearings, however, the reported burden was not re- 
vised downward. We were informed that defense burden estimates were 
recalculated after concluding that earlier defense expenditure estimates were 
too high. The burden did not change, it was claimed, because GNP growth 
was falling at the same time that defense growth was falling. 

Using test calculations based on the CIA’s pre-revision and post-revision 
figures, we were unable to obtain a Soviet defense burden in the range re- 
ported by the CIA in 1983 (13-14%) for any reasonable value for the rate 
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of growth of GNP for 1976 to 1981 (which the CIA reported in the 1982 
Hearings as 3.1% per year). We find the 1983 assertion that even after 
revision the burden remained 13-14% inconsistent with the CIA’s own 
earlier reported figures. There may be an explanation for the apparent in- 
consistency, but we have not been able to ferrit it out. The persistent in- 
ability of the CIA to explain this apparent discrepancy has further contributed 
to doubts about the candor of their reporting on defense matters.’ 

The 1986 Revision. The GNP and defense estimates before 1982 were 
valued at the prices of 1970. In 1982 the Soviets introduced a general price 
revision. The prices of 1970 had become increasingly unreflective of produc- 
tion costs in the years after 1970, and, following standard procedure, the 
CIA reevaluated both its GNP and defense estimates at the more meaning- 
ful prices of 1982. 

There is a solid body of economic theory leading one to expect that when 
a growth rate is evaluated at the prices of a later year it will prove to be 
smaller than when it is evaluated at the prices of an earlier year. In fact, 
most of the CIA’s results conform to this theoretical expectation. GNP, 
for example, grew at 3% per year at 1970 prices, but only at 2.6% at the 
new 1982 prices. 

However, the revised defense burden turns out to be higher at 1982 prices. 
Instead of the 13-14% reported for earlier years, the revised, 1982-price- 
base estimate jumps to 15-17%. We consider this a surprising result as 
yet inadequately explained by SOVA’s defense specialists. If the revised 
base prices do in fact measure true factor cost, then the burden of defense 
should decline in normal expectation. We suspect that the increase is an 
artifact of the method of estimating 1982 prices and/or making 1982 factor 
cost adjustments. 

We believe that the poor reliability of the defense share estimate raises 
a serious question about its usefulness and recommend that SOVA consider 
avoiding the calculation and publication of the numbers until more reliable 
numbers are available. In principle the size of the military burden is of in- 
terest because it provides a measure of how much the defense effort is cost- 
ing, and therefore how much it is crowding out the alternatives of 
consumption and investment in the U.S.S.R. In practice, however, virtu- 
ally noen of our analysis of Soviet policy depends on knowing the precise 
size of the burden. All one can assert is the self-evident proposition that 
the higher the burden, the harder it is for them to sustain their defense ef- 
fort. Nobody’s forecast of the Soviet Union’s ability or intention to sustain 
its military effort would be different if the burden were 15% than if it 
were 10% or 20%; we simply don’t know enough about the minds of the 
leadership or of the population to make such distinctions. We, therefore, 
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recommend that much less effort be devoted to the estimation of military 
expenditures, because cruder estimates with even a larger margin of un- 
certainty would be adequate. We also recommend that a burden estimate 
not be presented at all, on the ground that the estimate is extremely rough 
and that it contributes very little to the further understanding of the Soviet 
military threat. What is more, SOVA’s estimate of the burden has limited 
relevance for understanding the military burden as Soviet leadership views 
it. Presumably, a current ruble estimate would be the most appropriate 
measure. 

Finally, it should not be inappropriate for an intelligence agency to ac- 
knowledge that some information it would like to have is simply not avail- 
able. That acknowledgement might be conveyed by a statement such as the 
following: ‘“We have full confidence in our information on Soviet defense 
production, but the data vailable to the U.S. intelligence community are, 
in our opinion, insufficient to support any reliable estimate of Soviet defense 
spending by any known method of estimation.” 

Presentation. The CIA findings are thought to have had strong political 
repercussions, particularly in debates on what the proper level of U.S. 
defense expenditures should be. Although this is perfectly proper in a 
democracy, it serves as a powerful reminder that the findings should be 
reported in a way that would contribute to the most informed political judge- 
ment and public debate. The CIA cannot be held responsible for misinterpre- 
tation or misrepresentation of its findings, but it does have the responsibility 
to present its findings in a way that would minimize the likelihood that such 
things will occur. 

We have found that the CIA’s reporting of its findings on Soviet GNP 
and on general Soviet economic development has been satisfactory. With 
regard to the CJA’s findings on defense expenditures, however, we believe 
that the CIA presented their results in ways that contributed to misinterpre- 
tation and misrepresentation. For the most part, the effect of the form of 
presentation was to magnify the Soviet military threat to a level not war- 
ranted by the CIA’s own findings. 

Three cases will serve to illustrate the basis of our judgement. First, there 
is a sharp distinction to be drawn between the magnitude of the Soviet mili- 
tary threat and the level of Soviet defense expenditures. The magnitude of 
the military threat is fully conveyed by the CIA data on the size and quality 
of the Soviet armed forces. What share of GNP is required for the Soviet 
Union to mount that threat in no way affects the magnitude of the current 
threat; it relates at best to strategic issues. In fact, for a given magnitude 
of threat, the more the Soviets have to spend to support it, the stronger 
the position of the U.S., for it means that the Soviets are hurting more. 
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From this perspective, much of the huge effort in which the CIA has en- 
gaged to evaluate Soviet defense expenditures has not been worth the while. 
The vital information lies instead in the evidence obtained by covert and 
other means for assessing the Soviet order of battle, its command and con- 
trol capacilities, and other evidence of a military nature. In the CIA’s an- 
nual reporting it is that evidence that should be presented as its main 
contribution, for that is the basis on which our own defense effort should 
be based. Presumably this was the case in the CIA’s classified briefings. 
But the estimates of Soviet defense spending occupied the center of the public 
debate, deflecting attention there from the military threat itself. We do not 
know how the public debate may have influenced the private discussions 
as feedback. If the problem is one of classification, we believe the public 
debate is sufficiently important to warrant a change in policy. 

One example of the consequence, cited by Professor Franklyn Holzman, 
is the public response to the first major revision of the CIA’s defense esti- 
mates. In 1976, as we reported above, the CIA announced that its earlier 
estimates of the Soviet defense burden had been based on prices that they 
subsequently learned were incorrect. Using the new, corrected prices, their 
recomputation produced an estimate of the defense burden of 11-13%, in- 
stead of the 6~8% that had reported in the past. To be sure, the CIA ex- 
plained that both estimates were based on exactly the same number of planes, 
ships, and so forth; only the prices were different. Nevertheless the atten- 
tion given to that huge increase in reported defense expenditures caused 
President Nixon to write: ““Thanks in part to that intelligence blunder, we 
will find ourselves looking down the nuclear barrel in the 1980s.” 

Perhaps the CIA should not be held responsible for that kind of misin- 
terpretation. But we believe it could have reduced the impact of its defense 
expenditure findings by directing less attention to them and by more dili- 
gently hammering away at the irrelevance of the spending figure for the 
assessment of the current military threat. 

Second, in addition to its reporting of defense expenditures as a percen- 
tage of GNP (the burden), the CIA also regularly reported Soviet defense 
expenditures as a percentage of U.S. defense expenditures. Like the defense 
burden estimate, the comparative spending estimate deflects attention from 
the crucial question of the size of the military threat. 

Moreover, in preparing the estimate, the CIA frequently chose to report 
Soviet defense spending in dollars. Everyone engaged in the international 
comparison business knows that using dollar prices will show Soviet ex- 
penditures, whether on guns or butter, to be a larger proportion of the cor- 
responding U.S. expenditures than if the comparison were made in rubles. 
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To avoid this upward bias, the common practice is to prepare two such 
estimates, one in dollars and one in rubles. If a single final estimate is re- 
quired, the usual compromise is to report the average of the dollar estimate 
and the ruble estimate, although some dispute the meaningfulness even of 
this number. 

This accepted procedure is well-known to the CIA analysts, and it is in- 
deed the procedure sometimes followed in reporting comparative GNP es- 
timates. But in presenting comparative defense expenditures, the CIA reports 
have tended to emphasize the upward-biased dollar estimates and not the 
downward-biased ruble estimates or the average. The consequence is that 
the public image of the magnitude of Soviet defense spending relative to 
the U.S. was higher than it would have been if generally accepted best proce- 
dure were followed. We, therefore, applaud information that the CIA has 
ceased reporting comparative defense spending in dollars. However, if re- 
quests for comparative spending cannot be denied, the CIA should follow 
the standard practice of reporting both the dollar estimate and the ruble es- 
timate. 

Third, virtually all the CIA comparative defense reporting juxtaposes the 
Soviet Union and the U.S. If the objective is to provide the best evidence 
regarding the military threat, however, the most appropriate subjects of 
comparison should have been NATO and the Warsaw Pact. That form of 
reporting would be better for many reasons. The main reason is that, in 
our strategy for meeting the Soviet military threat, it was not U.S. military 
forces alone on which we counted but also the substantial forces of our 
NATO allies. A second reason is that in seeking to understand Soviet 
strategy, the U.S.S.R. should have been seen as facing not only U.S. mili- 
tary forces but those of Germany, Britain, and the other allies, including 
France in most circumstances. We are aware that the CIA did indeed pub- 
lish one classified document comparing NATO and Warsaw defense ac- 
tivities. The introduction states that it was to be the ‘‘first’’ such study (Sov 
87-10077). It gives a significantly more balanced picture of the Soviet mili- 
tary threat than that which emerges from the U.S.-Soviet comparison alone. 
Unfortunately, no subsequent studies have been undertaken so far as we 
know. 

We believe that by focusing on the U.S.S.R.-U.S. comparisons rather 
than on NATO, the great weight and authority of the CIA has served to 
bias the public debate on U.S. defense policy. Had greater attention been 
directed toward NATO-Warsaw Pact comparisons, the magnitude of the 
Soviet military threat would not have appeared as large as it appeared from 
the U.S.S.R.-U.S. comparisons alone. 
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Interpretive Reporting 


Apart from reports on defense expenditures, the review committee has 
formed a generally favorable opinion of the quality of the CIA’s interpre- 
tive reports on the Soviet economy. CIA’s reports on the growth of GNP 
and its components (excepting the defense component) during the 1980s 
were generally accurate and informative. We believe that the CIA’s clients 
received timely and reasonably accurate information also on the slowing 
rate of growth and the increasingly tough allocation decisions that Soviet 
leaders would face in the resulting resource crunch. There were, however, 
some significant ‘‘misses’’ too. Most significantly, the CIA failed to fore- 
see that perestroika and glasnost and certain policy blunders would lead 
to the sharp decline of the economy and to the breakup of the Soviet Union 
at the end of the 1980s. The question we explore in what follows is just 
how good were the CIA’s interpretive reports relative to those of the profes- 
sion of Soviet specialists at large? 

The short answer is that the CIA was not significantly better or worse 
than academic specialists. Both academic and agency analysts had mixed 
records, and the bottom line reveals that neither anticipated the intensity 
of potential ethnic separation or even considered the possibility that the com- 
munist party would lose legitimacy completely and that the Soviet political 
and economic system would disintegrate. It perhaps should be said that 
neither did Gorbachev and his advisors. 

It needs to be underscored at the outset that the actual course of events 
between 1988 and 1990 in the U.S.S.R. was, even in hindsight, a low prob- 
ability outcome. The problem for forecasters is that low probability events 
do nonetheless occur, and there is an indefinite number of low probability 
outcomes to consider. No forecaster can represent such possible outcomes 
as anything other than as of low probability. This offers no basis for criti- 
cism and illustrates the review committee’s preference for scenario style 
analysis of a range of possible future events of differing likelihoods, for 
it allows policy makers at least to consider the ramifications of several low 
as well as high probability outcomes. 

Regarding the CIA’s success in forecasting, the 1977 analysis of trends 
and prospects (ER 77-10436U) is an impressive piece of analysis. It cor- 
rectly identified 1976 as a critical inflection point in growth trends and 
projected accurately the potential long-run consequences of the slowdown. 
Several subsequent studies, notably two in 1981 (ER 81-10009 and ER 
81-10023), elaborated this theme helpfully. The report entitled ‘‘Soviet Per- 
ceptions of Economic Prospects’? (ER 81-10018) spelled out very neatly 
the rising Soviet concerns about energy resources, planning and management 
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deficiencies, and consumer welfare, all of which pointed to the need for 
a new growth strategy—one based upon intensive growth. The analysis of 
the futility implied by the ‘‘treadmill of reform’’ (Sov 82-10068) was right 
on the mark. Despite growing economic problems, no genuine economic 
reform was in fact taking place. Subsequent analyses, one (Sov 83-10093), 
based on a series of CIA industry studies that were particularly well chosen, 
provided a major contribution to our understanding of the causes and in- 
tractability of Soviet stagnation, a second (Sov 83-10202), provided an ex- 
cellent interpretation of the problems posed by excess demand, forced saving 
and the way in which the Soviet leadership had become ensnared in its own 
economic illogic. These reports illustrate the very best in economic analy- 
sis and forecasting and compare favorably with the best academic analyses 
of the period. 

Some analytic products do not fare as well in retrospect. The National 
Intelligence Estimate for 1982 (‘‘The Soviet Challenge to U.S. Security 
Interests’), for example, portrays the U.S.S.R. as aggressively confront- 
ing the United States in the Third World and as prepared to maintain the 
growth of its defense establishment at all costs. The growing economic 
stresses and strains receive little recognition in the analysis. One report (Sov 
84-10017) that did confront directly the larger implications of the slowing 
rate of economic growth proved to have been wrong in retrospect: 


. . . [T]he period of continued low level of growth that we project 
through 1990 should not be taken as a harbinger of economic col- 
lapse. . . . It [is] more accurate [to say that] our projection depicts 
a difficult and stressful period for a large, if inefficient, economy.”’ 


Academic specialists were not noticeably better in interpreting the ultimate 
consequences of the growth slowdown. But it is important to note also that 
the essence of this report is still informative: that a stressful period is being 
entered. Most students of the Soviet economy believed at that time that the 
Soviet economy would continue to “‘muddle through.’’ 

Joseph Berliner, for example, explored four possible scenarios for eco- 
nomic change (Reactionary, Conservative, Radical and Liberal), and ended 
his contribution to The Soviet Economy: Toward the Year 2000 (1983 edited 
by Levine and Bergson) with: 


I conclude that the prospects for change in the system of planning 
and management depends upon the performance of the economy under 
the Conservative Model. If the growth rate should stabilize at a level 
that may be low but that nevertheless exceeds the incentive threshold, 
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that model will be retained and the century will limp quietly to its 
end. Chronic cases do not normally evoke extreme measures. Only 
acute attacks, like depressions or rebellions, galvanize a society into 
such measures. If the Conservative Model cannot stabilize the growth 
rate even at that low level, the accumulation of social and political 
pressures will propel the leadership into either the Reactionary or the 
Liberal Model. Both are likely to improve the performance of the 
economy, but the greater potential lies with the latter. If the counsels 
of political prudence prevail, however, the lot will fall to the former. 


Berliner’s look into the future is more satisfying than the previous fore- 
cast because it is based upon alternative scenarios, but it still missed in- 
cluding the actual outcome because there was none that assumed the collapse 
of the Communist party. The Radical Model was a model of market so- 
cialism, not the repudiation of socialism. 

Seweryn Bialer in the same volume concluded: ‘‘In sum, we can antici- 
pate no fundamental changes in the Soviet Union during the 1980s despite 
intense and divisive discussions concerning economic reforms, a number 
of organizational policy initiatives, experimentation with the economic struc- 
ture, and significant political conflict.’’ Bialer and Joan Afferica argued 
in Foreign Affairs at about the same time (Winter 1982/83) against the no- 
tion that the U.S. had acquired a degree of leverage over domestic Soviet 
developments because of the deterioration of the economy. They declared 
this a ‘‘worst case’’ analysis: 


That severe economic stress will provoke political collapse may 
be a possible outcome in the next decade, but it is nonetheless an un- 
likely one. What generations have wrought with so much sacrifice, 
cruelty and conviction will not change radically under pressures of 
economic decline or leadership instability. The Soviet Union is not 
now nor will it be during the next decade in the throes of a true eco- 
nomic crisis, for it boasts enormous unused reserves of political and 
social stability that suffice to endure the deepest difficulties. The Soviet 
economy, like any gigantic economy administered by intelligent and 
trained professionals, will not go bankrupt. It may become less ef- 
fective, it may stagnate, it may even experience an absolute decline 
for a year or two; but, like the political system, it will not collapse. 


And Bialer was of essentially the same opinion three years later in The Soviet 
Paradox, External Expansion, Internal Decline (1986). 
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Analyzing the circumstances facing the Andropov regime, Walter Laqueur 
(‘‘What We Know About the Soviet Union,’’ Commentary, 1983) conceded 
the seriousness of the political and economic problems, but he argued that 
‘a modern dictatorship has powerful instruments with which to assuage, 
to suppress protest, to postpone the day of reckoning for a very long time.”’ 
Although he granted the existence of an economic crisis, he dismissed it 
as a possible cause of fundamental change in the Soviet system: ‘‘The long 
and short of it is that the Soviet economy will muddle through.”’ 

Richard Pipes (in Survival Is Not Enough, 1984) also failed to entertain 
the possibility that the party and economic bureaucracy might be completely 
dismantled. He cited Chalidze and Djilas approvingly ‘‘That reforms are 
conceivable only as a result of major internal and external setbacks, that 
they will come about only when the nomenklatura concludes that they are 
the price it must pay for its survival.’’ Pipes himself considered and dis- 
cussed only ‘‘what kind of reform can one expect from the Soviet leader- 
ship?’’ Even this was a low probability outcome in his view. How much 
more so must have been the actual course of Soviet history since 1988. 

Our purpose is not to single out individuals for criticism, but to docu- 
ment our contention that highly qualified academic scholars shared with 
those who worked for SOVA an assumption that the Soviet Union, despite 
its visible problems, remained fundamentally sound in the early and middle 
1980s. Many other scholars could be cited in support of this thesis. The 
review committee’s criticism of the work of SOVA in this connection is 
not for failing to see what essentially no one saw, but for its occasional 
failure to present its speculations in the form of explicit alternative scenarios 
with probabilities attached to each. 

One SOVA report (Sov 84-10104) was particularly categorical in dimiss- 
ing the possibility of radical change in the Soviet Union, and it may indeed 
have generated some of the criticism the CIA has received for its reports 
on the Soviet economy. The authors conclude that: 


[T]he chances are low that, by 1990, the growth slowdown will pre- 


cipitate . . . unrest . . . or compel [a] shift to a decentralized socialist 
market economy. . . . The slowdown is not likely to lead Moscow 
to forgo major weapons programs. . . . [N]or is it likely to substan- 


tially increase economically based Western political leverage, [or] 
drive the leadership to accept arms control agreements. . . . 


In conclusion, the report states that the ‘‘chances are low” that the eco- 
nomic slowdown will: 
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e Precipitate . . . widespread popular unrest in the U.S.S.R. 

e Pave the way for either significant liberalization or a Polish-style 
militarization of the regime. 

© Bring to power a leadership group with significantly different for- 
eign policy aims. 


The authors also conclude that ‘‘Moscow’s economic problems, while seri- 
ous and intractable, do not threaten the continued existence of the Com- 
munist political system.” 

Two years later, another SOVA report (Sov 86-10017X) provided a simi- 
larly conservative view of possible changes in the Soviet social system, but 
it was more successful in considering the alternatives. Like most Western 
analysts outside the U.S. Government, the authors concluded that major 
systemic change was unlikely. Rather, what was expected was ‘‘vigorous 
reform,” a slight increase in legal private activity, a shift in the balance 
of domestic political power to the KGB and an expanded role for the party. 
Overall, they concluded: “‘we expect that Gorbachev’s political position 
will remain strong and the U.S.S.R. .. . is likely to see some improve- 
ment in system performance over the next few years.’’ More radical re- 
form (a ‘‘new NEP’’) was rated a low probability scenario because it would 
be ‘‘risky and controversial.’’ A move toward greater repression was viewed 
as of low probability also and would result only out of leadership ‘‘desper- 
ation.’’ Most probably was a ‘‘No surprise-free future,’’ that is, ‘‘further 
unconventional leadership changes and surprises appear probable.’’ We con- 
sider this a model report under the circumstances of uncertainty that 
prevailed at the time it was prepared. It is not only informative, it also 
pointed in the correct direction and represents a good forecast. 

In 1987, a NIE report appeared, entitled ‘‘Whither Gorbachev: Soviet 
Policy and Politics in the 1990s,” also a solid piece of work in our opinion. 
This was the first general assessment of perestroika and glasnost, and the 
report stated that they contained the ‘‘potential’’ to change the U.S.S.R. 
as radically as Stalin had done in his time. The main question was whether 
or not Gorbachev had, or could, concentrate sufficient power in his hands 
to do so. This was a very valuable and timely analysis. (Incidentally, the 
director of NSA dissented from this conclusion.) The report concludes that: 
the most likely outcome of the Gorbachev reforms would be the rejuvena- 
tion of the existing system; there was a small chance (perhaps 1 in 3) to 
effect true systemic reform; a neo-Stalinist outcome was even less likely; 
and ‘‘the odds of a turn toward democratic socialism, featuring a more 
radical push for a market economy and a pluralistic society will remain 
virtually nil under any circumstances.”’ 
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Finally, and significantly, the report concludes: ‘‘We believe he 
[Gorbachev] recognizes he is pursuing a potentially dangerous course but 
is determined to try—even at the risk of failure—rather than accept the status 
quo.”’ 

The review committee believes that this is as good a report as its clients 
can hope to receive about a society in flux. Even those American scholars 
who were forecasting ‘‘gloom and doom’’ for the Soviet economy were 
not more accurate in their appraisals. Marshall Goldman, for example, in 
Gorbachev’s Challenge (1987), was dubious about the likelihood of suc- 
cess of Gorbachev’s early reforms, anticipating “‘improvements and im- 
proved productivity, but not the breakthrough Gorbachev seems eager to 
achieve.’’ Goldman expected the Soviet leadership to continue to ‘‘experi- 
ment,’’ and he speculated that ‘‘It is conceivable that the Soviets will adopt 
the Chinese approach, but it does not seem likely. Instead, the Soviet Union 
will probably continue to look to East Germany as a model of reform, 
making a marginal and even radical change here and there, and to operate 
much as it did in past periods of reform.’’ After considering the possibility 
of radical reform, Goldman decided that: 


The odds are that, given the resistance his reforms will face, and the 
risks a really radical reform entails, he [Gorbachev] will opt for the 
more conservative route. 

Short of some unexpected catastrophe, the Soviet economy is un- 
likely to come close to collapse, but without some equally unexpected 
transformation neither is it likely to improve its standing among the 
world’s economic leaders. In the end, Gorbachev, like his predeces- 
sors, will probably have to settle for an economy that has to rely more 
on its natural riches than on its creative potential. 


Finally, as late as 1988, Ed Hewett (in Reforming the Soviet Economy. 
Equality versus Efficiency) considered a middle course of reform as the most 
likely. He ruled out as very low probability both his ‘‘success scenario,” 
where ‘‘everything works as planned’’ and both qualitative and quantita- 
tive goals are met, and his ‘‘failure scenario,’’ which is essentially a ‘‘mud- 
dling through’’ outcome, not a collapse. The remaining alternatives, ‘‘high 
growth” and ‘‘high quality,’’ appear to have been equally probable in 
Hewett’s view. Thus, even at this late date so perceptive a student of the 
Soviet economy as Ed Hewett was still working within the old framework 
of analysis, one essentially the same as Gertrude Schroeder’s in the ‘“tread- 
mill of reform.” 
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We conclude, therefore, that the CIA’s interpretive reporting was at least 
as good as any competitor. There were some notable successes, and some 
notable misses, but overall the CIA’s reports have been of high quality, 
timely and useful to policy makers. We believe it would be unreasonable 
to expect otherwise unless CIA analysts were privy to critical secret sources. 
This does not appear to have been the case. 

The CIA’s assessment in the early 1980s, which was shared by the 
scholarly community generally, was that there was a very ‘‘low probability” 
that the system would collapse from popular unrest or from internal strains. 
Subsequent events have not proven that assessment to have been wrong, 
contrary to what many critics now assert. The system in fact did not col- 
lapse from internal strains and there was no public unrest. It collapsed for 
another reason that no one foresaw; namely, the cumulative sequence of 
events that culminated in the dissolution of the Party by the very man who 
had been elected five years earlier to lead the Party and had no intention 
at that time of contributing to its dissolution. 

The profession did identify the more probable outcomes, like ‘‘muddling 
down,”’ and some of the less probably outcomes, like the adoption of market 
socialism. But it did not foresee the possibility of one particular outcome, 
that the Party would be destroyed by the hand of the Party leader himself. 
In that sense the whole profession failed. 

Had that scenario been put forward for consideration, it would no doubt 
have been generally considered as of extremely low probability, and properly 
so. Low-probability events do occur, however, and the occurrence of such 
an event does not signify that those who had judged that event to be im- 
probable were wrong. Prophets are expected to foresee the most improb- 
able events, but social scientists do not have that power, should not be 
expected to have that power, and should be criticized when they claim to 
have that power. Good social science may be wrong, but it should be wrong 
for the right reasons. Only prophets should never be wrong. 


Notes 


1. Members of the Review Committee, brought together at CIA Headquarters 
in Langley, Virginia on July 26, 1991, by John Helgerson, Deputy Director for 
Intelligence, were briefed by George Kolt, John Helgerson and Judge William 
Webster on the instruction from the House Permanent Select Committee on Intelli- 
gence (HPSCI) to form a review panel. Public criticism, notably by Senator Moynihan 
and several prominent columnists, of the CIA’s assessment of Soviet economic and 
military capabilities during the 1970s and 80s indicated a concern that the CIA may 
have exaggerated the size and rate of growth of the Soviet economy and the Soviet 
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military threat, thereby causing, among other things, unnecessary defense expendi- 
tures by the United States. Deputy Director Helgerson indicated that HPSCI wants 
to know whether or not the CIA’s intelligence on the Soviet economy was ‘‘reason- 
able, timely and rang the gongs appropriately.’’ The committee was instructed to 
evaluate the appropriate materials and to deliver its report in November, 1991. 

2. The assumptions underlying this model are outlined in ‘‘SOVSIM: A Model 
of the Soviet Economy’’ (ER 79-10001). 

3. A 1979 paper, ‘‘Simulations of Soviet Economic Growth Until 1989” (ER 
79-10131), for example, used SOVSIM to examine the impact of forecasted labor 
and energy shortages in the 1980s. 

4. There were non-econometric analyses of growth problems conducted by SOVA 
during the Brezhnev era that were particularly useful and should be continued and 
perhaps expanded. The analysis by Gertrude Schroeder (a SOVA consultant), in 
““The Slowdown in Soviet Industry 1976-1982” (published subsequently in SOVIET 
ECONOMY), is an excellent example of the type and level of work we believe 
should be encouraged. 

5. The basic methodology for estimating the machinery component of Soviet and 
U.S. investments is spelled out, for example, in ‘‘U.S.S.R. and the U.S. Price Ratios 
for Machinery, 1967-1972,’’ Vol. I (ER 80-10410). 

6. According to ‘‘A Comparison of Soviet and U.S. GNPs 1960-83’’ (Sov 
84-10114), the Soviet economy gained on the U.S. between 1960 and 1975, rising 
from 49 to 58% (as a geometric average). It then declined to 55% between 1975 
and 1983. 

7. An instance that confirms our concern was the 1976 revision of the burden 
estimate, on the basis of new information about Soviet prices, from 6-8% of GNP 
to 11-13%. If an estimate can be off by almost 100% without any reliable inde- 
pendent evidence that it was so far off, one cannot have much confidence in the 
estimating procedure. Unfortunately, attempts to develop an independent estimate 
of defense expenditure have apparently not yielded sufficiently sensitive alternatives. 

8. In our opinion, this is an example of SOVA’s failure to do ‘‘a better job of 
explaining, documenting and qualifying the estimates,” as it was enjoined to do 
by the 1983 Working Group on Soviet Military Economic Analysis. 


